🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8127

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Unrelated Failures

As of commit 38d22fb with merge base 92e7dbd ():

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This pull request was exported from Phabricator. Differential Revision: D68847677

This PR needs a release notes: label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

let me explain a little on this, Tokens, freq_cos, freq_sin, mask, k, v caches will be passed in both prefill and decode model
The freq_cos /freq_sin/mask . Here we remove input_pos since prefill model inference will be like AR-pre_fill_len model , the input_pos does not come into play, runtime can fill freq_cos/freq_sin/kv/mask information directly. let me know about your thoughts, in case you have better ideas to support multi-turn, that will be great!